Object tracking device

ABSTRACT

A system includes a microcontroller, one or more sensors affixed to an object, and memory storing one or more programs including instructions for receiving and storing first sensor data from the one or more sensors in response to motion of the object, determining whether the first sensor data meets a first threshold, in accordance with a determination that the first sensor data meets the first threshold: receiving and storing second sensor data from the one or more sensors in response to subsequent motion of the object for as long as the second sensor data meets a second threshold, performing pattern recognition on the second sensor data, and identifying a first position of the object based on the pattern recognition of the second sensor data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/248,700 filed Jan. 15, 2019, which is a continuation of U.S. patentapplication Ser. No. 15/837,927 filed Dec. 11, 2017, now U.S. Pat. No.10,318,000 issued Jun. 11, 2019, which is a continuation of U.S. patentapplication Ser. No. 15/237,587 filed Aug. 15, 2016, now U.S. Pat. No.9,846,482 issued Dec. 19, 2017, which is a continuation of U.S. patentapplication Ser. No. 14/963,214 filed Dec. 8, 2015, now U.S. Pat. No.9,417,693 issued Aug. 16, 2016, which claims the benefit of U.S.Provisional Patent Application No. 62/089,216 filed Dec. 8, 2014, all ofwhich are incorporated by reference herein in their entirety.

TECHNICAL FIELD

The disclosed implementations relate to object tracking. Moreparticularly, the disclosed implementations relate to sensors, switches,control units, wireless communication modules and power sources,enabling users or devices to wirelessly communicate with and sendtracking parameters to electronic receivers and computing devices.

BACKGROUND

Most current tracking systems lack wireless capability; an object canonly move as far away from a receiving device as the length ofcables/wires will allow. The use of embedded hardware also contributesto the rigidity of such devices and reduces consumer appeal, while thereliance on custom communications protocols for many such devicesfurther hinders their wide adoption. There is an unmet need in themarket for tracking systems implementing communication protocolstandards for interfacing with electronics and computers, and which alsohave the ability to track positions and paths of objects using low powerimplementations.

SUMMARY

Various implementations of devices, apparatuses, and methods within thescope of the appended claims each have several aspects, no single one ofwhich is solely responsible for the attributes described herein. Withoutlimiting the scope of the appended claims, after considering thisdisclosure, and particularly after considering the section entitled“Detailed Description,” one will understand how the aspects of variousimplementations are used for creating context-based event entries.

In accordance with some implementations, a system includes amicrocontroller, one or more sensors affixed to an object, and memorystoring one or more programs including instructions for receiving andstoring first sensor data from the one or more sensors in response tomotion of the object, determining whether the first sensor data meets afirst threshold, in accordance with a determination that the firstsensor data meets the first threshold: receiving and storing secondsensor data from the one or more sensors in response to subsequentmotion of the object for as long as the second sensor data meets asecond threshold, performing pattern recognition on the second sensordata, and identifying a first position of the object based on thepattern recognition of the second sensor data. In accordance with someimplementations, a method includes one or more of the operationsdescribed above. In accordance with some implementations, anon-transitory computer readable storage medium stores one or moreprograms configured for execution by a computer system, the one or moreprograms including instructions for executing one or more of theoperations described above.

Various advantages of the present application are apparent in light ofthe descriptions below.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the aforementioned aspects of theinvention as well as additional aspects and implementations thereof,reference should be made to the Detailed Description below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIG. 1 is a dorsal view of the inner lining of the Gesture Glove.

FIG. 2 shows the relative positioning of all sensors, switches, batteryand electronic components of the Gesture Glove.

FIG. 3 illustrates the Positive (+) voltage wiring layer of the GestureGlove for Bend Sensors.

FIG. 4 illustrates the connections between the bend sensors and theMicrocontroller Unit (MCU).

FIG. 5 illustrates the Negative (−) voltage wiring layer of the GestureGlove for Switches.

FIG. 6 illustrates the connections between the pushbutton switches andthe MCU.

FIG. 7 illustrates the mounting positions of pushbutton switches on thefingers of the Gesture Glove.

FIG. 8 illustrates the primary layer of felt-type material of theGesture Glove.

FIG. 9 illustrates the wiring from the MCU (on top of the layer depictedin FIG. 8 above) to the switches and sensors (found underneath the topof the layer depicted in FIG. 8).

FIG. 10 illustrates the protection layer of thin felt-type materialapplied on top of the MCU and on top of the bend sensors.

FIG. 11 illustrates the final outer jacket of the Gesture Glove(encasing all composite layers below).

FIG. 12 is a partial see-through view of the overall assembled GestureGlove, showing all its substrata of electronics and fabric liningspartially visible under the outer jacket.

FIG. 13 is an isometric view of the finished Gesture Glove with itsouter jacket.

FIG. 14 is a surface rendered view of the finished Gesture Glove.

FIG. 15 illustrates the functional structure of a novel bend sensordeveloped for the Gesture Glove.

FIG. 16 is a flow diagram of the processing steps performed by the MCUof the Gesture Glove.

FIG. 17 is a flow diagram of the processing steps performed by the RFreceiver device (e.g., console application or controllable electronicdevice).

FIG. 18 is a generalized representation of the processing steps toextract the individual gesture positions and hand orientation by thereceiver device.

FIG. 19 is a flow diagram of the processing steps performed by anapplication for Mouse & Joystick control by the receiver computer.

FIG. 20A is a block diagram of a wearable human machine interface (HMI)device of which the Gesture Glove is an implementation and an electronicdevice controllable by the wearable HMI device.

FIG. 20B is a block diagram that depicts an implementation similar toFIG. 20A, but with the Gesture Library and Gesture Detection located onthe HMI (as opposed to FIG. 20A where the Gesture Library and GestureDetection are located on the controllable device itself).

FIG. 21 is a block diagram showing functional details of the wearableHMI device of FIG. 20A.

FIG. 22 is a block diagram showing functional details of an electronicdevice controllable by the wearable HMI device of FIG. 20A.

Like reference numerals refer to corresponding parts throughout theseveral views of the drawings.

DETAILED DESCRIPTION

Numerous details are described herein in order to provide a thoroughunderstanding of the example implementations illustrated in theaccompanying drawings. However, some implementations may be practicedwithout many of the specific details, and the scope of the claims isonly limited by those features and aspects specifically recited in theclaims. Furthermore, well-known methods and systems have not beendescribed in exhaustive detail so as not to unnecessarily obscure morepertinent aspects of the implementations described herein.

Referring now to the invention in more detail, FIG. 1 illustrates thedorsal view of the inner lining 100 of an example implementation of aGesture Glove. In some implementations, a comfortable blend of polyesteror cotton fabric is used for the inner lining 100. TheMetacarpophalangeal joint 101 and the Proximal Interphalangeal joint 102are also delineated in FIG. 1, for positional reference of sensors andelectronic components.

FIG. 2 is a dorsal see-through view of an example implementation of theGesture Glove, showing the relative positioning of joint sensors 110,111, pushbutton switches 120, 121, 122, Microcontroller (MCU) 130,Motion Processing Unit (MPU) 131, Radio Frequency Transmission Unit(RFTX) 132, and the power source (e.g., a battery pack or one or morelithium polymer batteries, such as of the rechargeable type) 133. Forgreater clarity, the words “switch” or “switches” throughout thisdisclosure shall mean, without limitation any of: positional switches,directional switches, directional controls, miniature joysticks, etc. Insome implementations, in addition to the sensors and switches describedherein, the Gesture Glove may include one or more joy sticks, infraredsensors, global positioning system sensors, or any similar sensorsconfigured to detect user gestures or user inputs. In someimplementations, positions and numbers of the sensors, switches,electronic components (e.g., the MCU, MPU, RFTX and power source) andalternative input devices (e.g., joy sticks, etc.) are different inwhole or in part from the positions and numbers of those components asillustrated in FIG. 2. Such differences can be due to packagingconstraints of different implementations (e.g., room available for thepower supply), or operational and/or ergonomic constraints applicable tothe sensors and/or user input devices (e.g., user-friendly positioningof a joystick).

Four bend sensors 110 are placed on the dorsal side of the ProximalInterphalangeal joint 102, and one bend sensor 111 is placed between thethumb and the index finger. Additional bend sensors (similar to bendsensor 111) may be optionally placed between each of the other fingers.Four switches are mounted obliquely, on the Medial Phalanx of eachfinger 120. The index finger contains an additional switch on theProximal Phalanx 121. Two more switches are mounted on the dorsalsurface of the hand 122, at the base of the thumb, for auxiliary controlfunctions.

FIG. 3 illustrates the Positive (+) voltage wiring layer 112 used todrive the bend sensors 110 and 111. Each bend sensor receives the sourcevoltage with a parallel wiring schema.

FIG. 4 illustrates the connection of individual bend sensor 110, 111signal wires 113 with the Microcontroller Unit (MCU) 130. Each bendsensor is monitored individually by the MCU software for state change.

FIG. 5 illustrates the Negative (−) voltage wiring layer 123 used todrive pushbutton switches 120, 121, 122. Each pushbutton switch islinked to the source voltage with a parallel wiring schema. Eachpushbutton switch may provide a digital signal to the MCU indicatingthat a pushbutton is “pressed” or not “pressed”.

FIG. 6 illustrates the connection of individual pushbutton switches 120,121, 122 signal wires 124 to the microcontroller unit (MCU). Eachpushbutton switch position is monitored by the MCU software for statechange.

FIG. 7 illustrates the oblique mounting of pushbutton switches on theMedial Phalanges 120, a typical arrangement for four fingers, and theadditional pushbutton switch 121 on the Proximal Phalanx of the indexfinger parallel to the sagittal plane. This orientation preventsaccidental actuation of switches while flexing fingers.

FIG. 8 illustrates the primary layer of soft, thin, felt-type material140 over the wiring stratum. Wires run under the fabric overlay. A slotor opening is provided in the middle of the fabric 141 for the wiring topenetrate the top side of the fabric for connectivity with theirrespective destination points.

FIG. 9 illustrates the connection of communication wiring 141 to the MCUmounted on top of the felt type fabric 140. The fabric layer protectsand isolates wiring underneath and electronics modules above (MPU, MCU,BATTERY, RFTX).

FIG. 10 illustrates the final layer of thin felt type material 150, 151,152 to protect the electronics modules (MPU, MCU, RFTX), and bendsensors 110, 111, respectively.

FIG. 11 illustrates the mounting of the final outer jacket (160) overthe composite layers below; the final external jacket is made of acompression fit fabric to finish the glove. Additional packing material161, such as felt, may be used to fill in and balance protrusion pointsof the glove at the Phalanges caused by embedded pushbutton switchesmounted obliquely on the Medial-Phalanges of fingers 120.

FIG. 12 illustrates the overall assembly of the gesture glove with itssubstrata of electronics and fabric linings partially visible under theouter jacket 160.

FIG. 13 is an isometric view of the finished Gesture Glove with theouter jacket 160.

FIG. 14 is a surface rendered view of the finished Gesture Glove using acompression fit material.

FIG. 15 illustrates the functional structure of a typical bend sensor110, 111 used for detecting finger bend position in the Gesture Glove.This bend sensor is mounted over the Dorsal side of the ProximalInterphalangeal joint. The bend sensor is constructed with a core 170 ofsemi-conductive carbon infused resilient plastics, such as polyurethaneor polyethylene porous substrate material, capable of being optionallyimpregnated with one or more of: ethanol, isobutane, propane,1,1-difluroethane, powdered carbon and powdered graphite. Above andbelow the core 170, there are two thin, flexible electrically conductiveplates 171 (made of a conductive and corrosion-resistant sheet material)that complete a conductive pathway over the entire upper and lowersemi-conductor core surface. In an implementation, wider electricallyconductive plates 171 are used for optimal sensing; a wide surface ofcontact with the electrically conductive layers reduces signal noise andovercomes local signal degradation caused by surface irregularitiesproduced during the manufacturing process. The wide contact surface areaalso provides current pathway redundancy, fault tolerance, and greateraccuracy and reproducibility between different batches of productionunits or sensors that have the same overall dimensions. Any knownelectrically conductive means of adhesion (such as brazing, welding, orsoldering) 172 can be used to attach conductive wire leads 173 to theupper and lower electrically conductive plates 171. Polarity of appliedcurrent is interchangeable between the upper and lower electricallyconductive plates 171. The electrically conductive plates 171 along withthe core semi-conductive layer 170 and attached wire leads 173 areenclosed in a flexible and durable thin plastic sheathing 174. Thesheathing is manufactured with a flat tapered region 175, which acts asthe designated surface for fastening the sensor assembly to the rest ofthe Gesture Glove (any fastening means applied to region 175 will notinduce compression of the core sensing areas 170 and 171). At the sensorend opposite to the region 175, the area where the wire leads 173 areexiting the sheathing 174 is sealed with a non-conductive epoxy glueseal 176, to prevent core slippage, and to provide an anchor point forthe wire leads in order to prevent disconnection or fatigue at theirrespective attachment points 172 on the electrically conductive plates171 from repeated movements. The epoxy seal 176 also serves as a secondfastening surface for the bend sensor. Bending this sensor assembly ineither direction along the length of the sensor will stretch andcompress the semi-conductive material, increasing the conductivedensity, and reducing the resistance of the sensor assembly.

In some implementations, the carbon infused porous substrate may befurther impregnated with electrical-resistance altering chemicaladditives such as ethanol, isobutane, propane, 1,1-difluroethane,powdered carbon, powdered graphite, etc.). To this end, the poroussubstrate may be impregnated by being soaked in various solutions orsuspensions containing ethanol, isobutane, propane, 1,1-difluroethane,powdered carbon, powdered graphite, followed by low heat drying for onehour or more, such as 24 hours, so as to achieve various weight % levelsof chemical additive impregnation. In some implementations, the poroussubstrate material has an absorptive sponge-like and absorptiveconsistency with high permeability to absorb the chemical solution.

In different implementations, materials and chemical concentrations canbe adjusted to produce the desirable dynamic range for resistance changeas needed. For example, a higher carbon deposition will produce higherconductivity and smaller dynamic range. Alternatively, a higher chemicaladditive deposition will produce higher resistance when material is notbent, and greater dynamic range during flexion. Thus, the range can beadjusted between unbent to bent flexion. For example, themaximum-minimum resistivity range may be about 30 k ohms to about 2 kohms respectively in some implementations with approximately 10%chemical deposition within the porous substrate, and about 200 k ohms toabout 5 k ohms respectively with 100% chemical deposition within theporous substrate in other implementations.

In operation, the sensor may be positioned in a wearable HMI device inproximity to a finger or hand joint or a muscle of a user of the HMIdevice, such that any motion of such joint or muscle causes deformationof the sensor, resulting in the sensor outputting directly to the MCU arespective analog signal representative of extent of the deformation.The analog signal varies because the bend sensor has a variableresistance that changes relative to the amount of bending or flexing ofits physical components. In some implementations, a bend sensor (e.g.,bend sensors 110 or 111) provides a full range signal (e.g., about 1volt to about 3.3 volts) to communicate bend sensor position to acomputing device such as a microcontroller (e.g., MCU 130 in FIG. 2). Insome implementations, a bend sensor exhibits a wide resistance rangefrom about 2,000 ohms at a 90 degree bend to about 30,000 ohms in astraight neutral position using a 1.5 mm thick, low-density, carboninfused polyurethane porous substrate 170 impregnated with chemicaladditives at 10% per weight. This range eliminates the need for externalsignal conditioning circuitry, permitting a bend sensor to directlyinterface with a microcontroller and thereby reduce latency. In someimplementations, the bend sensor may operate at an applied sourcedriving voltage as low as 3.3 volts.

FIG. 16 is a flow diagram of the processing steps performed by the MCUof the Gesture Glove.

At step 1601, the MCU initializes the system. The system may includeelectronic modules such as (MPU, MCU, BATTERY, RFTX). Examples ofinitializing the system may include loading device settings or defaultsfor certain electronic modules and setting thresholds to determine whento communicate sensor data.

Steps 1601 a-1601 d describe a process for initializing a systemaccording to at least one implementation of the invention. At step 1601a, the MCU loads system communication and device settings. At step 1601b, the MCU loads defaults to communications module. At step 1601 c, theMCU initializes the MPU. At step 1601 d, the MCU sets thresholds formotion sensors and for bend sensors. When a value generated by a motionsensor and a bend sensor exceeds the threshold set by the MCU, the MCUwill transmit the sensor data measured by one or more sensors, such as abend sensor) to another computing device.

At step 1602, the MCU samples data from one or more sensors, such as abend sensor, pushbutton switch, miniature joystick or MPU. As discussedherein, the sensors may be configured to detect orientation and/orposition and/or movement and/or flexion of a user and generate sensordata corresponding to such orientation and/or position and/or movementand/or flexion.

At step 1603, the MCU determines whether the sensor data meetspredetermined transmission criteria by comparing the sampled data fromeach sensor to a threshold. If the sampled data does not exceed thethreshold, the MCU returns to step 1602 and continues to sample datafrom the one or more electronic devices. If the sampled data exceeds thethreshold, the MCU advances to step 1604. In some implementations, theMCU may sample data from all sensors when the sampled data from one ormore sensors exceeds the threshold. By determining whether sensor datameets predetermined transmission criteria, the MCU can judiciouslydecide whether to transmit data or refrain from transmitting data.Transmitting data requires power and with the HMI operating on a fixedpower supply to allow a user to have free range of movement, conservingpower provides a better overall experience for the user.

At step 1604, the MCU averages the sampled data with statisticalconditioning based on whether the MCU is operating in a low latency orhigh accuracy mode.

In a low latency mode, a single sensor to measure each specificorientation or movement of the user may be used. By using a singlesensor, less processing time is required, meaning lower latency isrealized overall by the system. In these implementations for low latencyapplications, the raw sensor data for gyroscopic, accelerometric, andmagnetometric data may be associated on a one-to-one basis with a sensoroutput (one gyroscope, one accelerometer, one magnetometer) using anyknown filtering algorithms or fusion algorithms (e.g., a Kalman filteror a custom filter). The sensor data is then aggregated over multiplereadings and time averaged to produce a steady output with very lowdrift and low noise. A low pass filter algorithm employed after thetime-averaged data stream is sampled minimizes noise yielding verystable orientation and motion data that can be used to control a consoleapplication efficiently and accurately. Time-averaging prior to the lowpass filtering is critical in achieving the high accuracy in output.

In a high accuracy mode, the MCU may receive sensor data from multipleredundant sensors (e.g., two or more accelerometers to measure the samelinear direction or two or more gyroscopes to measure the same angularmovement). Redundant sensors may be helpful because inertial sensors areprone to long-term drift, a type of noise that manifests itself in thesensor data. By averaging the sampled data from multiple redundantsensors, the MCU can reduce the amount of noise in the sampled data. Forexample, if the sampled data includes multiple samplings, it is possiblethat some of the samplings may have a substantial noise component.However, if other samplings have a limited noise component, theaveraging of the samplings will reduce the overall noise component inthe sampled data. By reducing noise, the electronic devices describedherein can more easily process the desired signal component of thesampled data.

In some implementations, the multiple redundant sensors may be mountedat fixed and known distances relative to each other on a custom printedcircuit board. The distances may be predetermined during circuit boardfabrication. The sensor data from the multiple redundant sensors is usedwith each output being subjected to filtering methodology in a lowlatency mode, as described above. The positional and orientation datafrom each sensor is then considered with the known actual physical andorientation data to further eliminate drift, noise, and position foroptimum accuracy. This configuration minimizes long-term drift and noisedeviation because the actual distances and angles between sensorssoldered on the MPU board are already known.

In a hybrid low-latency high-accuracy mode, the MCU may include a 20 Mhzto 50 Mhz processor. In this implementation, the high accuracy and lowlatency methods described above can be used with a nominal trade-off ofhigher battery consumption. In some implementations, multiple lowfrequency MCUs can be connected in tandem using I2C or SPI communicationbus to collect and process data from each set of gyroscopes,accelerometers, and/or magnetometers to divide processing load andaccomplish higher throughput with similar trade-off of higher batteryconsumption.

In some implementations, the processes performed by the MCU in steps1603 and 1604 may be performed in reverse. In these implementations, theMCU averages the sampled data to reduce noise and then compares thesampled data to a threshold.

At step 1605, the MCU creates an ASCII data stream as a name=value pairof the sampled data from each sensor. In some implementations, the namecorresponds to the electronic component (e.g., a bend sensor or pushbutton switch) and the value corresponds to a human action or gesturemeasured by the electronic component. For example, “B0=H” where “B0” isthe Push-button Switch 121 located on the Proximal Phalanx of the indexfinger mounted perpendicularly and “=H” represents the value of “B0”where “H” is the “pressed” button state. The value of zero (0) as in“B0=L” would mean the Push-button is not “pressed”. In someimplementations, each name and/or value is represented by 8 bytes percharacter. The use of an ASCII data stream and name=value pair isconducive to use of an application platform interface (API) tocommunicate with a console application. By using ASCII data, anydeveloper programming a console application or similar electronic devicecan easily interpret and process this data format making the gestureprocessing system universal or device agnostic.

In some implementations, the MCU creates an encoded (e.g., compressed)data stream. In these implementations, each byte of sample data isencoded to a corresponding value for further processing and transmissionto a console application. One of the benefits of encoding data is toreduce the amount of data transmitted between the glove and a consoleapplication, thereby improving gesture processing latency.

At step 1606, the MCU transmits sensor data from one or more sensors toa console application (i.e., a controllable device) using a wirelessmodule (e.g., Radio Frequency Transmission Unit (RFTX) 132).

In some implementations, the MCU transmits the data as a datatransmission packet. The data transmission packet is made up of i)standard ASCII characters describing named variables and theirrespective values or ii) encoded values corresponding to the sensordata. The full packet containing all data variables, including switches,bend sensors, and motion processing unit of the HMI, is then pushedthrough a Radio Frequency Transmission Unit (RFTX) 132, a Bluetooth orWiFi (IEEE 802.1) module.

In some implementations, the MCU transmits the sensor data from at leasttwo (optionally, all) sensors to a console application if sensor datafrom at least one sensor exceeds a threshold as described in step 1603,thus achieving improved accuracy

At step 1607, the MCU resets all variables, registers and countersstored in its memory and reverts back to step 1602 to continue samplingsensor data.

FIG. 17 is a flow diagram of the processing steps performed by the RFReceiver Device (e.g., unmanned aerial vehicle, gaming console,controllable device, console application). FIG. 17 depicts the primarylayer, class, or process to intercept data from the Gesture Glove. Thislayer can be used on any electronic or computing device. The output ofthis layer is represented by “A” at the bottom of FIG. 17 which is anevent forwarded to higher layers or encapsulating classes. Name valuepairs are converted into arrays or lists that are looped through andparsed to assess the variable states, such as Pushbutton Switches 120,121, 122, Bend Sensor Positions 110, 111, as well as hand-orientationand movement (MCU) 131. If any variables exceed their given threshold,an event “A” is raised for higher-layer applications of encapsulatedclasses; such as Gesture Control applications and Mouse Controlapplications described in FIGS. 18 and 19 respectively. By processingdata by the RF Receiver Device, the MCU requires less processing powerto operate, thereby preserving battery power. Also, the RF receiverdevice may have faster processing capabilities, thereby reducing latencyby interpreting gesture commands at the receiver device as compared tointerpreting gesture commands at the MCU. However, it is contemplatedthat, in some alternative implementations, such as that shown in FIG.20B, it is the MCU that processes the sensor data, interprets thegestures and generates gesture commands. In such implementations, theMCU transmits the gesture command to the RF receiver device. The gesturecommands, when executed by a console application, cause the consoleapplication to perform an action (e.g., cause an unmanned aerial vehicleto perform a barrel roll). The MCU may use similar techniques asdescribed in step 1605 and step 1606 above (e.g., ASCII name=value pairor compression encoding techniques) to transmit the gesture commands tothe RF receiver device.

FIG. 18 is a generalized representation of the processing steps toextract the individual gesture positions and hand orientation in animplementation of this invention. The event received from the output “A”as described in FIG. 17, is assessed for values exceeding a thresholdlimit for each variable. If the values are below the threshold, then theprocess terminates or exits in FIG. 18. If the threshold is exceeded,then the ongoing inputs subsequent to that trigger are stored in memoryup until the moment the values fall below the threshold for the givenvariable. Subsequently, the data collected is traced and subjected to apattern recognition algorithm, such as Hidden Markov Model or a NeuralNetwork with integrated Fuzzy Logic, the output of which identifies asaid motion that triggers an action, macro, stored procedure, program,etc.

FIG. 19 is a flow diagram of the processing steps performed by anapplication for Mouse & Joystick control. The event received from theoutput “A” as described in FIG. 17 is parsed for hand-orientation andmotion that are converted into Mouse movements and Pushbutton statesthat emulate various Mouse-button states in the operating system of thereceiver computer.

In more detail, still referring to the invention of FIGS. 1-19, theGesture Glove uses wireless transmission to control electronic devicesusing multiple methods. In an implementation, there are three methods ofcontrolling a device wirelessly through the Gesture Glove. Firstly, thebend orientation of fingers 110 and thumb 111 can be used as an inputmethod to control a wirelessly connected device. Secondly, embeddedswitches 120 or alternate user controls (e.g., a joystick or capacitivetouch pad) can be used to control a linked device. Thirdly, the handorientation and motion (MPU) 131 can also be used as gestural controlparameter. Furthermore, any combination of these three methods can beused for a plurality of additional gesture controls. This novel mixtureof input methods combined with a new and universal data-transmissionpacket, described in detail below, makes the Gesture Glove a uniquesolution as a multi-functional human-machine interface device.

One of the most distinct and unique functional features of the GestureGlove is the method in which the Bend Sensor 110 functions. In someimplementations, four (4) Bend Sensors 110 measure the bend position ofthe four (4) fingers at the Proximal Interphalangeal joint 102 on thedorsal side of each finger. The fifth (5th) Bend Sensor 111 is placedbetween the thumb and the index finger. When the fingers are straight(not bent) the sensors are not compressed; therefore, the conductivedensity of core material 170 is lowest and resistance of the sensor ishighest in pathway current flow pathway between the two (2) electricallyconductive plates 171. This resistance, depending on the bend position,and varying between 5,000Ω (90 degree bend) to 200,000Ω (straight,un-bent position), is sampled and averaged over several readings in thetime-order of micro-seconds and relayed to the wireless receiver devicethrough the wireless Radio Frequency Transmission Unit (RFTX) 132.

A second method of control that the Gesture Glove implements is throughthe use of Push-button Switches 120. The Microcontroller Unit (MCU) 130monitors and relays two (2) states (ON or OFF) of the PushbuttonSwitches 120 to a given wireless receiving device during a data-packettransmission.

A third distinct method of control that the Gesture Glove utilizes isthrough the use of hand orientation and motion. To facilitate this, aninertial Motion Processing Unit (MPU) 131 is used to measure the staticG-Force on the XYZ axis. These values are read and transmitted by theMicrocontroller Unit (MCU) 130 as the tilt-orientation of the hand tothe wireless receiver device within a wireless data-packet transmission.The spatial motion of the hand is measured using the same MPU 131. Amovement in any axial plane causes inertial force on the MPU 131 that isalso read and relayed by the MCU 130 within a wireless data-packettransmission. For gesture detection, the Gesture Glove is configurableto perform the following:

-   -   i) static positional gesture detection, such as detecting a        value of 5 units (five minutes or five points) when the user        raises and holds still his be-gloved hand with all his five        fingers spread apart;    -   ii) simplistic movement gesture detection, such as interpreting        a simple motion of the hand in the air to mean swatting, or        throwing, or “crossing out:’    -   iii) combined gesture detection, such as assigning a certain        meaning to a certain movement of the hand performed while        simultaneously holding the fingers in a certain position.

To measure orientation and spatial motion of the hand, the MPU 131 mayinclude one or more sensors such as magnetometers, gyroscopes, inertialsensors, accelerometers and electro-myograms. The magnetometers maymeasure the orientation of a body part of a user relative to the Earth'smagnetic field. The gyroscopes may measure changes in angularorientation of a body part of a user in one or more axes. Theaccelerometers may measure changes in movement of a body part of a userin one or more (e.g., three) axes. The electro-myograms may measureelectrical signals produced during muscle contractions by a user. TheMPU 131 may be configured to detect tension or stress levels on the skinof the user and encode the sensor data to correspond to a generalconfiguration of the body part (e.g., hand) of the use. In suchconfigurations, the MPU 131 may detect tension on the palm of the user,or detect if a hand of a user is in a first or has fingers extended.

In some implementations, the MPU 131 may include redundant pairs (e.g.,two, four, six) of sensors, such as magnetometers, gyroscopes,accelerometers and electro-myograms. In these configurations, the MPU131 may reduce sensor noise by incorporating redundancy and averagingthe sensor data from groups of redundant sensors. By averaging thesensor data from groups of redundant sensors, anomalous noise found inone sensor can be minimized by averaging the sensor data with anomalousnoise with sensor data from properly functioning sensors.

In some implementations, the MPU 131 may include a multi sensor array toreduce drift or inaccuracies/noise in the sensor data that accumulatesover time.

In some implementations, the MPU 131 is a custom printed circuit boardhaving an array of sensors. In some implementations, the MPU 131includes a multi-master, multi-slave, single-ended, serial computer busto transmit data between sensors and the MCU 130. The MCU 130 mayprocess sensor data from the MPU 131 using the I2C protocol developed byPhilips Semiconductor®. In some implementations, each sensor isaddressable and the bus contains two outputs, a clock signal and thesensor data. At each clock cycle, the MCU 130 samples one bit of datafrom each sensor until all sensor data is sampled. Then, the MCU 130repeats the sampling process.

Once a data-packet is received by the wireless receiver device, it issubjected through a cascade of processing layers, each designed tocouple or decouple compatibly, making the Gesture Glove universallyadaptable for a broad range of field application with minimal add-onelectronics or computational power. FIG. 17 describes a process forintercepting the data-packet and pre-processing the values of eachvariable transmitted by the Gesture Glove before being passed on tohigher layers of processing. This method establishes a base layer forcommunication that can be standardized in a form of software class,importable into encapsulating classes that used the standard proceduresof the primary layer to intercept the Gesture Control data and buildmore complex control applications. As an example, a higher level classor procedure is described in FIG. 18 to detect a predefined gesture orhand movement to trigger a program, method, macro, etc. further up thechain. Another application for mouse control is described in FIG. 19.Data received by the computer via Bluetooth, radio frequency (e.g.,amplitude modulation, frequency modulation, phase modulation) or WiFi isparsed and converted to represent positional X, Y values for the mouse,as well as the button state can be mapped to emulate mouse movements andclicks on the resident Operating System of the computer. Additionally,the gesture control application and mouse control application can becascaded together to create even more robust applications with amultiplicity of controls through combinatorial use of finger, switch,and motion sensors in the Gesture Glove. Moreover, Left-Hand andRight-Hand Gloves can be used in concert for even greater control of thereceiver device.

In further detail, still referring to the invention of FIGS. 1-19, theGesture Glove can be made in several commercial sizes (e.g., Small,Medium, and Large) to fit the most common hand size ranges for adultsand children.

With the optional use of compression fit materials of improvedelasticity, it can be possible to produce the Gesture Glove in justseveral pre-set sizes that would still cover most hand sizes ofpotential users.

As to its construction, any suitable glove making technique known in theart can be used to assemble the Gesture Glove. In one implementation ofthis invention, as shown in FIGS. 1-19, is assembled in layers to makethe Gesture Glove thin, durable, and practical. Beginning with the first(innermost) layer, the inner lining 100 is made of a soft andcomfortable blend of cotton or polyester or any other suitable material.Mounted on top of the inner lining 100, there are bend sensors 110, 111,pushbutton switches 120, 121, 122, and their respective power andcommunication wiring 112, 113, 123, 124. A layer of felt-type material140 (or any other suitable material) is introduced above this wiringlayer for isolation and protection for the electronics layer to follow.A slit, orifice or cut-out 141 is made in the centre of this felt-typematerial 140 to allow lower stratum wiring to emerge on the electronicslayer placed on top of the felt-type material as shown in FIG. 9.Microcontroller Unit (MCU) 130, Motion Processing Unit (MPU) 131, RadioFrequency Transmitter (RFTX) 132 and Battery 133, with their respectivecommunications and power wiring are located in the electronics layer.The electronics and the bend sensors are then covered with another layerof felt-type material (applied in areas 150, 151, 152 of FIG. 10) toprotect them against physical damage during use and to provide a smoothcontour or finish to the glove by dampening any protrusions caused bythe electronics, sensors, switches, and wiring. Finally, a compressionfit material in the form of a glove is placed on top of all layers belowto create a finished glove. In one implementation, the flexibility andfeel of the final glove assembly are very close to an off-the-shelf highquality, high-performance glove with a snug fit. In anotherimplementation, a custom-fabricated scaffolding, inner skeleton orexo-skeleton may be fitted to the Gesture Glove or may replace one ormore or all layers of the Gesture Glove, so as to provide support forthe electronic components affixed to it. In yet another implementation,the electronic components of this invention may be directly and/orindividually attached to the fingers, thumb, palm or back-of-the-hand ofthe user by any known means, with or without one or more coveringlayer(s) resembling a garment glove.

The advantages of various implementations of the present inventioninclude, without limitation, its plug & play capability owing to itsstandard protocol and base class software. The Gesture Glove provides,in its various implementations, multiple control inputs permittinghundreds of distinct gestures. The ASCII data-packet transmissionimplementation allows rapid and easy development of applications thatcan use the Gesture Glove as the input interface to a large variety ofdevices. The slim, flexible appearance gives the user a feeling ofwearing a regular glove. Some implementations employ similar dataprocessing principles and components in different wearable HMI deviceform factors, such as a fingerless glove, a sleeve that fits around oneor multiple fingers or around another body part, such as one or more ofa wrist, forearm, elbow, ankle, foot or calf.

In some implementations, the present invention is a wearable devicewhich enable its users the ability to interface with electro-mechanicaland electronic devices in a more dynamic and natural way.

FIG. 20A is a block diagram of master-slave environment 2000 including awearable human machine interface (HMI) device 2001 of which the GestureGlove is an implementation and an electronic device 2020 controllable bythe wearable HMI device in accordance with some implementations. In someimplementations, the master-slave environment 100 includes an HMI device2001 (e.g., Gesture Glove) coupled to an electronic device 2020 (e.g.,an unmanned aerial vehicle, a console application, an RF receiverdevice) and communication network(s) 2010 for interconnecting thesecomponents. Examples of the one or more networks 2010 include local areanetworks (LAN) and wide area networks (WAN) such as the Internet. Theone or more networks 2010 are, optionally, implemented using any knownnetwork protocol, including various wired or wireless protocols, such asEthernet, Universal Serial Bus (USB), FIREWIRE, Global System for MobileCommunications (GSM), Enhanced Data GSM Environment (EDGE), codedivision multiple access (CDMA), time division multiple access (TDMA),Bluetooth, Wi-Fi, voice over Internet Protocol (VoIP), Wi-MAX, or anyother suitable communication protocol.

In some implementations, the HMI device 2001 includes sensor(s) 2002and/or a microcontroller 2004. The sensor(s) 2002 is configured todetect orientation and/or position and/or movement and/or flexion of theuser (i.e., a gesture) and generate sensor data corresponding to theorientation and/or position and/or movement and/or flexion. Themicrocontroller 2004 is configured to sample sensor data from sensor(s)2002 and determine whether to transmit the sensor data or correspondinggesture control data to the electronic device 2020. As described herein,the microcontroller 2004 can be configured to operate in differentmodes, such as a low latency mode and a high accuracy mode, and transmitcontrol data to the electronic device 2020 based on the mode. Thecontrol data is data that, when processed by the electronic device 2020,causes the electronic device 2020 to perform an action (e.g., perform amouse click function or steer an unmanned aerial vehicle).

In some implementations, the electronic device 2020 can include anapplication 2022 and/or a gesture library 2024, and/or an IntelligentGesture Detection algorithm 2026. The application 2022 is configured toprocess the control data received from the HMI device 2001 and execute acommand that causes the electronic device 2020 to perform an action. Insome implementations, the control data is sensor data. In theseimplementations, the application 2022 analyzes the sensor data andretrieves a gesture command from the gesture library 2024 based on thesensor data. The gesture library 2024 is a database of gesture commandsthat correlate to sensor data. An example of a gesture library 2024 isshown in FIG. 19 in the section titled “EXECUTE COMMAND CASCADE BASED ONSENSOR STATE”. The Intelligent Gesture Detection algorithm 2026 allowsthe intelligent detection of a gesture even when the gesture library2024 does not have any pre-set gesture commands that correlate to sensordata; instead, new gestures can be detected by intelligent processing,guessing and/or calculation on-the-fly. As shown in FIGS. 20A and 20B,the Gesture Library and the Intelligent Gesture Detection can resideeither on the controlled device itself (as in FIG. 20A) or on the HMIdevice (as in FIG. 20B).

In some implementations, the control data transmitted from the HMIdevice 2001 is a gesture command. In these implementations, the gesturelibrary 2024 may exist on the HMI device 2001. In these implementations,the microcontroller 2004 may analyze the sensor data and retrieve agesture command from the gesture library 2024 based on the sensor data.Once the gesture command is transmitted to the electronic device 2020,the application 2022 executes the command and causes the electronicdevice 2020 to perform an action.

FIG. 21 is a block diagram showing functional details of the wearableHMI device 2001 of FIG. 20A.

In some implementations, the HMI device 2001 includes control processingunits CPU(s) such as the microcontroller 2002 (i.e., MCU) shown in FIG.20, an MPU 2102, one or more network interfaces or other communicationsinterfaces 2104 (e.g., RFTX), user interface 2110, memory 2106, and oneor more communication buses 2108 for interconnecting these components.Communication buses 2108 may include circuitry (sometimes called achipset) that interconnects and controls communications between systemcomponents.

In some implementations, the HMI device 2001 includes one or more motionsensors 2116, which directly or indirectly determine the orientation andmovement of a body part of the user. In some implementations, thelocation/motion sensors 316 include, but are not limited to, gyroscopes,accelerometers, and GPS devices. In some implementations, the HMI device2001 may also include one or more bend sensors 2110, which determine theflexion of a body part of the user. In some implementations, the HMIdevice 2001 may also include one or more push button sensor(s) 2112,which detect user-initiated push of a button. The sensor data producedby any of the above sensors, in any combination, may be used by the HMIdevice 2001 to track user gestures and control the electronic device2020 based on gesture commands that correspond to the tracked usergestures.

Memory 2106 includes high-speed random access memory, such as DRAM,SRAM, DDR RAM or other random access solid state memory devices; andoptionally includes non-volatile memory, such as one or more magneticdisk storage devices, optical disk storage devices, flash memorydevices, EPROM, EEPROM or other known non-volatile solid state storagedevices. Memory 2106 optionally further includes one or more storagedevices remotely located from the CPU(s) 2002 (e.g., cloud storage).Memory 2106, or alternately the non-volatile memory device(s) withinmemory 2106, comprises a non-transitory computer readable storagemedium. In some implementations, memory 2106 or the computer readablestorage medium of memory 2106 stores the following programs, modules anddata structures, or a subset or superset thereof:

-   -   operating system 2120, which includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   network communication module 2122 (or transmitting module),        which is used for connecting the HMI device 2001 to electronic        device 2020 via the e.g., RFTX 2104 (wired or wireless) and one        or more networks 2010 (FIG. 20A), such as the Internet, other        wide area networks, local area networks, metropolitan area        networks, and so on;    -   sampling module 2124 for sampling the sensor data from the        plurality of sensors 2216, 2110 and 2112;    -   determining module 2126 for determining whether the sensor data        from one of the plurality of sensors meets transmission        criteria;    -   generating module 2127 for generating control data based on the        sensor data from a portion of the plurality of sensors when        operating in the low latency mode and generating control data        based on the sensor data from all of the plurality of sensors        when operating in the high accuracy mode; and    -   selecting module 2128 for selecting a gesture command to the        controllable device based on the sensor data from the plurality        of sensors.

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices of the HMI device 2001, andcorresponds to a set of instructions for performing a function describedabove. The above identified modules or programs (i.e., sets ofinstructions) need not be implemented as separate software programs,procedures, modules or data structures, and thus various subsets ofthese modules may be combined or otherwise re-arranged in variousimplementations. In some implementations, memory 2106, optionally,stores a subset of the modules and data structures identified above.Furthermore, memory 2106, optionally, stores additional modules and datastructures not described above.

FIG. 22 is a block diagram showing functional details of an electronicdevice 2020 controllable by the wearable HMI device 2001 of FIG. 20A. Insome implementations, the electronic device 2020 includes one or moreprocessing units (CPUs) 2204, one or more network interfaces or othercommunications interfaces 2205, memory 2206, and one or morecommunication buses 2208 for interconnecting these components.Communication buses 2208 may include circuitry (sometimes called achipset) that interconnects and controls communications between systemcomponents.

In some optional implementations, the electronic device 2020 alsoincludes an user interface that presents a graphical user interface(GUI) using one or more output devices and receives user inputs via oneor more input devices. In some optional implementations, the electronicdevice 2020 includes a user interface 2210 comprising zero or moredisplay devices 2212 (e.g., a screen or monitor) and zero or more inputdevices or mechanisms 2214. The one or more output devices, optionallyincluding one or more visual displays and optionally including one ormore speakers, enable presentation of media content by the electronicdevice 2020. In some implementations, the one or more input devicesinclude user interface components that facilitate user input, such as akeyboard, a mouse, a voice-command input unit or microphone, a touchscreen display, a touch-sensitive input pad, a camera, a gesturecapturing camera, and/or other input buttons or controls, and optionallyinclude two or more of such input devices. Furthermore, the electronicdevice 2020 may use a microphone and voice recognition, or a camera andiris/face recognition to supplement or replace a physical or virtualkeyboard.

Memory 2206 includes high-speed random access memory, such as DRAM,SRAM, DDR RAM or other random access solid state memory devices; andoptionally includes non-volatile memory, such as one or more magneticdisk storage devices, optical disk storage devices, flash memorydevices, EPROM, EEPROM or other known non-volatile solid state storagedevices. Memory 2206 optionally further includes one or more storagedevices remotely located from the CPU(s) 2204 (e.g., cloud storage).Memory 2206, or alternately the non-volatile memory device(s) within thememory 2206, comprises a non-transitory computer readable storagemedium. In some implementations, the memory 2206 or the computerreadable storage medium of the memory 2206 stores the followingprograms, modules and data structures, or a subset or superset thereof:

-   -   operating system 2220, which includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   network communication module 2222, which is used for connecting        electronic device 2020 to HMI device 2001 via the one or more        communications network interfaces 2205 (wired or wireless) and        one or more networks 2010 (FIG. 20A), such as the Internet,        other wide area networks, local area networks, metropolitan area        networks, and so on;    -   application module 2224 for analyzing sensor data (i.e., control        data) from HMI device 2001 to determine a corresponding gesture        command and/or executing gesture commands to cause electronic        device 2020 to perform an action;    -   gesture library 2230 for storing gesture commands and        corresponding sensor data and processing requests for gesture        commands from application module 2224.

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices of the electronic device 2020,and corresponds to a set of instructions for performing a functiondescribed above. The above identified modules or programs (i.e., sets ofinstructions) need not be implemented as separate software programs,procedures, modules or data structures, and thus various subsets ofthese modules may be combined or otherwise re-arranged in variousimplementations. In some implementations, memory 2206, optionally,stores a subset of the modules and data structures identified above.Furthermore, memory 2206, optionally, stores additional modules and datastructures not described above.

In some implementations, at least some of the functions of theelectronic device 2020 are performed by the HMI device 2001, and thecorresponding sub-modules of these functions may be located within theHMI device 2001. Furthermore, in some implementations, at least some ofthe functions of the HMI device 2001 are performed by the electronicdevice 2020, and the corresponding sub-modules of these functions may belocated within the electronic device 2020. The HMI device 2001, and theelectronic device 2020 shown in FIG. 20 are merely illustrative, anddifferent configurations of the modules for implementing the functionsdescribed herein are possible in various implementations.

It is contemplated that, using the devices and techniques describedherein, a user can employ any of the control modalities (buttons,fingers, motions, sensors, switches, miniature joysticks, etc.) of HMIdevice 2001 to define hundreds of custom gestures and implement hundredsof custom gesture commands that cause the electronic device 2020 toperform an action, when executed. This is possible due to the precisionallowed by the disclosed processing techniques, redundant sensors and/oraccuracy the disclosed flex sensors, among other reasons.

To create custom gestures, a gesture glove console application executedon the HMI device or another electronic device can be placed into alearning mode, during which repeated trials of a custom gesture areperformed by the user/operator wearing the gesture glove.

In current best-case applications using industry standard patternrecognition algorithms, approximately one hundred trials of same gesturepattern are required to establish a good statistical data set for agiven custom gesture in order to reduce data variance and estimate aproper decision boundary for reuse of the gesture pattern in a fieldapplication. These techniques are still under research and development.

The gesture glove console application, in contrast, requires less thanten training repetitions for custom gestures to be properly recognizedby the pattern recognition algorithm. This is accomplished by analgorithm or equation that uses weighted statistical measures includinga combination of partial algorithms from K-Nearest Neighbors, DynamicTime Warping, Support Vector Machines, and Non-Linear Correlationinfused in a new calculative methodology that enables a betterprediction of a decision boundary with fewer training sets for a customgesture. The predictive output from this methodology can be useddirectly for low latency applications, or further subjected toadditional recognition layers such as neural networks, decision trees,or hidden Markov for high accuracy applications.

Once the custom gesture is appended to the gesture library, the gesturelibrary can be used from the console application or uploaded to thegesture glove MCU for direct field use.

In one implementation, there is a wearable gesture control interfaceapparatus for controlling a controllable device based on gesturesprovided by a user, said wearable gesture control interface apparatusincluding: a plurality of sensors configured to detect one or moreparameters of the user, including orientation, movement, position,flexion, and generate sensor data corresponding to said one or moreparameters of the user; and a microcontroller configured to: sample,using a sampling module, the sensor data from the plurality of sensors;determine, using a determining module, whether the sensor data from oneof the plurality of sensors meets transmission criteria; and inaccordance with a determination that the sensor data from the one of theplurality of sensors meets the transmission criteria, transmit, using atransmitting module, control data corresponding to all of the pluralityof sensors to the controllable device.

In a further implementation, the transmission criteria is met when thesensor data from the one of the plurality of sensors exceeds athreshold.

In a further implementation, the control data includes the sensor datafrom the plurality of sensors.

In a further implementation, the control data includes gesture commandsthat, when executed by the controllable device, cause the controllabledevice to perform an action.

In a further implementation, the microcontroller is further configuredto select, using a selecting module, a gesture command to thecontrollable device based on the sensor data from the plurality ofsensors.

In a further implementation, the plurality of sensors and themicrocontroller are configured to mount to a glove attachable to a handof the user.

In one implementation, there is a method for controlling a controllabledevice based on gestures provided by a user, including: in a wearablegesture control interface apparatus having a microcontroller, aplurality of sensors, one or more functional modules and a memorystoring programs for execution by the microcontroller: performingoperations of the wearable gesture control interface apparatus of any ofthe above implementations.

In one implementation, there is a non-transitory computer readablestorage medium including a memory for use in a wearable gesture controlinterface apparatus having a microcontroller, a plurality of sensors andone or more functional modules, the memory storing programs that, whenexecuted by the microcontroller, cause the wearable gesture controlinterface apparatus to operate as set out in any of the aboveimplementations.

In one implementation, there is a wearable gesture control interfaceapparatus for controlling a controllable device based on gesturesprovided by a user, said wearable gesture control interface apparatusincluding a plurality of sensors configured to detect one or moreparameters of the user, including orientation, movement, position,flexion, and generate sensor data corresponding to said one or moreparameters of the user; and a microcontroller operable in a low latencymode and a high accuracy mode, the microcontroller configured to:sample, using a sampling module, the sensor data from the plurality ofsensors; generate, using a generating module, a sensor output based onthe sensor data from a portion of the plurality of sensors whenoperating in the low latency mode; and generate, using the generatingmodule, a sensor output based on the sensor data from all of theplurality of sensors when operating in the high accuracy mode.

In a further implementation, generating the sensor output includesaveraging the sensor data from the plurality of sensors to minimize anoise component in the sensor data.

In a further implementation, each of the plurality of sensors thatcomprise the portion of the plurality of sensors has a differentfunction and wherein at least two of the plurality of sensors areconfigured to measure an identical orientation or movement.

In a further implementation, a number of plurality of sensors isuser-selectable after the plurality of sensors are connected to amicrocontroller.

In a further implementation, each of the plurality of sensors areseparated on a circuit board at fixed predetermined distances relativeto another of the plurality of sensors.

In a further implementation, the microcontroller generates the sensoroutput based on the fixed predetermined distances separating theplurality of sensors.

In one implementation, there is a method for controlling a controllabledevice based on gestures provided by a user, including in a wearablegesture control interface apparatus having a microcontroller, aplurality of sensors, one or more functional modules and a memorystoring programs for execution by the microcontroller: performingoperations of the wearable gesture control interface apparatus of any ofthe above implementations.

In one implementation, there is a non-transitory computer readablestorage medium including a memory for use in a wearable gesture controlinterface apparatus having a microcontroller, a plurality of sensors andone or more functional modules, the memory storing programs that, whenexecuted by the microcontroller, cause the wearable gesture controlinterface apparatus to operate as set out in any of the aboveimplementations.

In one implementation, there is a sensor for measuring flexing orbending of an object, including a first flexible electrically conductiveplate and a second flexible electrically conductive plate; a flexiblesemi-conductive porous substrate disposed between and contacting thefirst and second electrically conductive plate, wherein a resistance ofthe semi-conductive porous substrate varies based on an amount offlexion of the semi-conductive porous substrate; a first lead connectedto the first flexible electrically conductive plate, the first leadconfigured to receive a drive signal from a microcontroller, the drivesignal being altered based on the resistance of the semi-conductiveporous substrate to generate an output signal; and a second leadconnected to the second flexible electrically conductive plate, thesecond lead configured to transmit the output signal directly to themicrocontroller without requiring additional signal conditioning.

In a further implementation, the semi-conductive porous substrate has avariable resistance of about 1,000 ohms to about 30,000 ohms based onthe amount of flexion of the semi-conductive porous substrate.

In a further implementation, the semi-conductive porous substrate has avariable resistance of about 5,000 ohms to about 200,000 ohms based onthe amount of flexion of the semi-conductive porous substrate.

In a further implementation, the semi-conductive porous substrate has avariable resistance of about 50,000 ohms to about 1,000,000 ohms basedon the amount of flexion of the semi-conductive porous substrate.

In a further implementation, the output signal has a range of about 1volt to about 4 volts.

In a further implementation, the output signal has a range of about 1.25volts to about 3.73 volts.

In a further implementation, the sensor is positioned in a wearable HMIdevice in proximity to a finger or hand joint or a muscle of a user ofthe HMI device, such that any motion of such joint or muscle causesdeformation of the sensor, resulting in the sensor outputting directlyto the MCU a respective analog signal representative of extent of thedeformation.

In a further implementation, the sensor is positioned in the wearableHMI device in proximity to a finger joint of the user.

In a further implementation, the sensor is positioned in the wearableHMI device in proximity to an area of a user's hand between two fingers,such that relative movement between the two fingers causes deformationof the sensor.

In one implementation, there is a wearable gesture control interfaceapparatus for controlling a controllable device based on gesturesprovided by a user, said wearable gesture control interface apparatusincluding a plurality of sensors configured to detect one or moreparameters of the user, including orientation, movement, position,flexion, and generate sensor data corresponding to said one or moreparameters of the user; and a microcontroller operable in a low latencymode and a high accuracy mode, the microcontroller configured to:sample, using a sampling module, the sensor data from the plurality ofsensors; determine, using a determining module, whether the sensor datafrom one of the plurality of sensors meets transmission criteria; and inaccordance with a determination that the sensor data from the one of theplurality of sensors meets the transmission criteria, themicrocontroller configured to: generate, using a generating module,control data based on the sensor data from a portion of the plurality ofsensors when operating in the low latency mode; and generate, using thegenerating module, control data based on the sensor data from all of theplurality of sensors when operating in the high accuracy mode; andtransmit, using a transmitting module, the control data to thecontrollable device.

In a further implementation, the transmission criteria is met when thesensor data from the one of the plurality of sensors exceeds athreshold.

In a further implementation, the control data includes gesture commandsthat, when executed by the controllable device, cause the controllabledevice to perform an action.

In a further implementation, the microcontroller is further configuredto select, using a selecting module, a gesture command to thecontrollable device based on the sensor data from the plurality ofsensors.

In a further implementation, the wearable gesture control interfaceapparatus further including a plurality of switches controllable by auser of the apparatus.

In a further implementation, the sensors comprise one or more flexsensors affixed to locations on the wearable gesture control apparatusthat can be deformed by movements of a user of the apparatus, whereinthe movements include movements of one or more finger or hand joints.

In a further implementation, generating the control data includesaveraging the sensor data from the plurality of sensors to minimize anoise component in sensor data.

In a further implementation, each of the plurality of sensors thatcomprise the portion of the plurality of sensors has a differentfunction and wherein at least two of the plurality of sensors areconfigured to measure an identical orientation or movement.

In a further implementation, a number of plurality of sensors isuser-selectable after the plurality of sensors are connected to amicrocontroller.

In a further implementation, each of the plurality of sensors areseparated on a circuit board at fixed predetermined distances relativeto another of the plurality of sensors.

In a further implementation, the microcontroller generates the sensoroutput based on the fixed predetermined distances separating theplurality of sensors.

In a further implementation, when the microcontroller operates in thelow latency mode, the controllable device is one of: a gaming console,computer joystick and a computer mouse.

In a further implementation, when the microcontroller operates in thehigh accuracy mode, the controllable device is one of: an unmannedaerial vehicle and sign language software.

In a further implementation, when the microcontroller operates in thehigh accuracy mode, the microcontroller is configured to track hundredsof hand gesture based on a combination of any of: at least one inertialmeasurement sensor, at least one flex sensor, and at least one pushbutton switch.

In a further implementation, the control data includes sensor data in aname value pair format.

In one implementation, there is a method for controlling a controllabledevice based on gestures provided by a user, including in a wearablegesture control interface apparatus having a microcontroller, aplurality of sensors, one or more functional modules and a memorystoring programs for execution by the microcontroller: performingoperations of the wearable gesture control interface apparatus of any ofthe implementations.

In one implementation, there is a non-transitory computer readablestorage medium including a memory for use in a wearable gesture controlinterface apparatus having a microcontroller, a plurality of sensors andone or more functional modules, the memory storing programs that, whenexecuted by the microcontroller, cause the wearable gesture controlinterface apparatus to operate as set out in any of the aboveimplementations.

In one implementation, there is a method for controlling a controllabledevice based on user gestures, including: in a wearable gesture controlinterface apparatus having a microcontroller, a plurality of sensors, atransmitting module and a memory storing programs for execution by themicrocontroller: sampling sensor data from the plurality of sensors inresponse to the gestures; determining whether the sensor data from oneof the plurality of sensors meets transmission criteria; and inaccordance with a determination that the sensor data from the one of theplurality of sensors meets the transmission criteria, transmitting usingthe transmitting module control data corresponding to sensor data fromall of the plurality of sensors to the controllable device.

In a further implementation, the sensors comprise one or more of flexsensors; user actuatable switch sensors, including or more of buttons,sliders, switches, joystick and touch pad; and one or more motionsensors, including one or more of gyroscopes, magnetometers andaccelerometers.

In a further implementation, the plurality of sensors and themicrocontroller are mounted to a glove or to a portion thereofattachable to a hand of the user.

In a further implementation, the transmission criteria are met when thesensor data from the one of the plurality of sensors exceeds apredetermined threshold.

In a further implementation, the control data include gesture commandsthat, when executed by the controllable device, cause the controllabledevice to perform one or more respective actions associated with thegestures.

In a further implementation, including selecting and transmitting to thecontrollable device a gesture command based on the sensor data from theplurality of sensors.

In one implementation, there is a method for controlling a controllabledevice based on gestures provided by a user, including in a wearablegesture control interface apparatus having a microcontroller, aplurality of sensors and a memory storing programs for execution by themicrocontroller: detecting using the plurality of sensors one or moreparameters of the user, including orientation, movement, position,flexion; generating sensor data corresponding to the one or moreparameters; determining an operating mode of the wearable gesturecontrol device; sampling the sensor data from the plurality of sensors;generating a sensor output based on the sampled sensor data from aportion of the plurality of sensors when the operating mode is a lowlatency mode; and generating the sensor output based on the sensor datafrom all of the plurality of sensors when the operating mode is a highaccuracy mode.

In a further implementation, the sensors comprise one or more of flexsensors; user actuable switch sensors, including or more of buttons,sliders, switches, joystick and touch pad; and one or more motionsensors, including one or more of gyroscopes, magnetometers andaccelerometers.

In a further implementation, the plurality of sensors and themicrocontroller are mounted to a glove or to a portion thereofattachable to a hand of the user.

In a further implementation, generating the sensor output includesaveraging the sensor data from the plurality of sensors to minimize anoise component in the sensor data.

In a further implementation, each of the plurality of sensors thatcomprise the portion of the plurality of sensors has a differentfunction and wherein at least two of the plurality of sensors areconfigured to measure an identical orientation or movement.

In a further implementation, each of the plurality of sensors areseparated on a circuit board at fixed known distances relative toanother of the plurality of sensors.

In a further implementation, the method further including generating thesensor output based on the fixed predetermined distances separating theplurality of sensors.

In one implementation, there is a method for controlling a controllabledevice based on gestures provided by a user, including in a wearablegesture control interface apparatus having a microcontroller, aplurality of sensors and a memory storing programs for execution by themicrocontroller: detecting using a plurality of sensors one or moreparameters of the user, including orientation, movement, position,flexion; generating sensor data corresponding to the one or moreparameters; detecting an operating mode of the wearable gesture controlinterface apparatus; sampling the sensor data from the plurality ofsensors; determining whether the sensor data from one of the pluralityof sensors meets transmission criteria; and in accordance with adetermination that the sensor data from the one of the plurality ofsensors meets the transmission criteria: generating control data basedon the sensor data from a portion of the plurality of sensors when theoperating mode is a low latency mode; and generating the control databased on the sensor data from all of the plurality of sensors when theoperating mode is a high accuracy mode; and transmitting the controldata to the controllable device.

In a further implementation, the transmission criteria is met when thesensor data from the one of the plurality of sensors exceeds apredefined threshold.

In a further implementation, the control data includes one or morerespective gesture commands that, when executed by the controllabledevice, cause the controllable device to perform an action associatedwith the one or more respective gesture commands.

In a further implementation, the method including selecting from agesture database the one or more respective gesture commands based onsensor data from the plurality of sensors, wherein the gesture databasesstores associations between respective gesture commands and sensor data.

In a further implementation, the wearable gesture control interfaceapparatus further includes a plurality of switches controllable by auser of the apparatus.

In a further implementation, the sensors comprise one or more flexsensors affixed to locations on the wearable gesture control apparatusthat can be deformed by movements of a user of the apparatus, whereinthe movements include movements of one or more finger or hand joints.

In a further implementation, generating the control data includesaveraging the sensor data from the plurality of sensors to minimize anoise component in the sensor data.

In a further implementation, each of the plurality of sensors thatcomprise the portion of the plurality of sensors has a differentfunction and wherein at least two of the plurality of sensors areconfigured to measure an identical orientation or movement.

In a further implementation, a number of the plurality of sensors isuser-selectable after the plurality of sensors are connected to amicrocontroller.

In a further implementation, each of the plurality of sensors isseparated on a circuit board at fixed known distances relative toanother of the plurality of sensors.

In a further implementation, the method further including generating thesensor output based on the fixed known distances separating theplurality of sensors.

In a further implementation, when the microcontroller operates in thelow latency mode, the controllable device is one of: a gaming console,computer joystick and a computer mouse.

In a further implementation, when the microcontroller operates in thehigh accuracy mode, the controllable device is one of: an unmannedaerial vehicle and sign language software.

In a further implementation, the method further including, when theoperating mode is the high accuracy mode, tracking hundreds of handgestures based on a combination of any of: at least one inertialmeasurement sensor, at least one flex sensor, and at least one pushbutton switch.

Each of the methods described herein is typically governed byinstructions that are stored in a computer readable storage medium andthat are executed by one or more processors of one or more servers orclient devices. The above identified modules or programs (i.e., sets ofinstructions) need not be implemented as separate software programs,procedures or modules, and thus various subsets of these modules will becombined or otherwise re-arranged in various implementations.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific implementations. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theimplementations were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best utilize the invention andvarious implementations with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. A method comprising: at a system including amicrocontroller, one or more sensors affixed to an object, and a memorystoring programs for execution by the microcontroller: receiving andstoring first sensor data from the one or more sensors in response tomotion of the object; determining whether the first sensor data meets afirst threshold; in accordance with a determination that the firstsensor data meets the first threshold: receiving and storing secondsensor data from the one or more sensors in response to subsequentmotion of the object for as long as the second sensor data meets asecond threshold; performing pattern recognition on the second sensordata; and identifying a first position of the object based on thepattern recognition of the second sensor data.
 2. The method of claim 1,further comprising performing pattern recognition on at least a portionof the first sensor data, wherein identifying the first position isfurther based on the pattern recognition of the portion of the firstsensor data.
 3. The method of claim 1, further comprising: in accordancewith the determination that the first sensor data meets the firstthreshold: receiving and storing third sensor data from a second sensorof the one or more sensors in response to the subsequent motion;performing pattern recognition on the third sensor data; and identifyingthe first position of the object based on the pattern recognition of thesecond and third sensor data.
 4. The method of claim 1, furthercomprising: in accordance with the determination that the first sensordata meets the first threshold: receiving and storing fourth sensor datafrom the one or more sensors in response to additional subsequentmotion; performing pattern recognition on the fourth sensor data; andidentifying a second position of the object based on the patternrecognition of the fourth sensor data.
 5. The method of claim 4, furthercomprising: tracing the first position and the second position of theobject; and storing a path of the object based on the tracing.
 6. Themethod of claim 1, wherein the pattern recognition includes one or morerecognition layers including at least one of: K-Nearest Neighbors,neural networks, decision trees, or hidden Markov.
 7. A systemcomprising: one or more microcontrollers, one or more sensors affixed toan object, and memory storing one or more programs to be executed by theone or more microcontrollers, the one or more programs includinginstructions for: receiving and storing first sensor data from the oneor more sensors in response to motion of the object; determining whetherthe first sensor data meets a first threshold; in accordance with adetermination that the first sensor data meets the first threshold:receiving and storing second sensor data from the one or more sensors inresponse to subsequent motion of the object for as long as the secondsensor data meets a second threshold; performing pattern recognition onthe second sensor data; and identifying a first position of the objectbased on the pattern recognition of the second sensor data.
 8. Thesystem of claim 7, further comprising instructions for performingpattern recognition on at least a portion of the first sensor data,wherein identifying the first position is further based on the patternrecognition of the portion of the first sensor data.
 9. The system ofclaim 7, further comprising instructions for: in accordance with thedetermination that the first sensor data meets the first threshold:receiving and storing third sensor data from a second sensor of the oneor more sensors in response to the subsequent motion; performing patternrecognition on the third sensor data; and identifying the first positionof the object based on the pattern recognition of the second and thirdsensor data.
 10. The system of claim 7, further comprising instructionsfor: in accordance with the determination that the first sensor datameets the first threshold: receiving and storing fourth sensor data fromthe one or more sensors in response to additional subsequent motion;performing pattern recognition on the fourth sensor data; andidentifying a second position of the object based on the patternrecognition of the fourth sensor data.
 11. The system of claim 10,further comprising instructions for: tracing the first position and thesecond position of the object; and storing a path of the object based onthe tracing.
 12. The system of claim 7, wherein the pattern recognitionincludes one or more recognition layers including at least one of:K-Nearest Neighbors, neural networks, decision trees, or hidden Markov.13. A non-transitory computer readable storage medium storing one ormore programs configured for execution by a computer system, the one ormore programs including instructions for: receiving and storing firstsensor data from one or more sensors affixed to an object in response tomotion of the object; determining whether the first sensor data meets afirst threshold; in accordance with a determination that the firstsensor data meets the first threshold: receiving and storing secondsensor data from the one or more sensors in response to subsequentmotion of the object for as long as the second sensor data meets asecond threshold; performing pattern recognition on the second sensordata; and identifying a first position of the object based on thepattern recognition of the second sensor data.
 14. The non-transitorycomputer readable storage medium of claim 13, further comprisinginstructions for performing pattern recognition on at least a portion ofthe first sensor data, wherein identifying the first position is furtherbased on the pattern recognition of the portion of the first sensordata.
 15. The non-transitory computer readable storage medium of claim13, further comprising instructions for: in accordance with thedetermination that the first sensor data meets the first threshold:receiving and storing third sensor data from a second sensor of the oneor more sensors in response to the subsequent motion; performing patternrecognition on the third sensor data; and identifying the first positionof the object based on the pattern recognition of the second and thirdsensor data.
 16. The non-transitory computer readable storage medium ofclaim 13, further comprising instructions for: in accordance with thedetermination that the first sensor data meets the first threshold:receiving and storing fourth sensor data from the one or more sensors inresponse to additional subsequent motion; performing pattern recognitionon the fourth sensor data; and identifying a second position of theobject based on the pattern recognition of the fourth sensor data. 17.The non-transitory computer readable storage medium of claim 16, furthercomprising instructions for: tracing the first position and the secondposition of the object; and storing a path of the object based on thetracing.
 18. The non-transitory computer readable storage medium ofclaim 13, wherein the pattern recognition includes one or morerecognition layers including at least one of: K-Nearest Neighbors,neural networks, decision trees, or hidden Markov.